Shared & Flexible Block I/O for Cluster-Based Storage

High-performance storage systems are evolving from centralized architectures and specialized hardware to ”storage bricks”, i.e. a large set of decentralized commodity components with more processing power and network throughput. These emerging systems offer increased flexibility for tailoring storage to application needs. In this paper, we present Locus, an extensible framework for cluster storage virtualization and sharing at the block-level. Locus allows to build customized storage systems by composing hierarchies of virtual storage devices on top of distributed physical disks. Locus hierarchies may be distributed almost arbitrarily over storage nodes and application servers, introducing significant freedom in mapping functions to available resources. Furthermore, Locus allows sharing of storage volumes at the block-level by providing block-locking and block-allocation services as modules that may be inserted in virtual hierarchies. To demonstrate the benefits of such an enhanced block interface, we show that it can substantially simplify the design of higher-level storage services, such as distributed (cluster) file systems. We implement the Locus framework and Locus-fs (a stateless, pass-trough file-system) under Linux and evaluate them over various setups using both single and multiple client and storage nodes. We find that the flexibility offered by Locus introduces little overhead beyond mandatory communication and disk access costs. Furthermore, experiments with a cluster of 16 nodes show that Locus scales well both at the block and file-system level.